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Application/Control Number: 09/934,004 
Art Unit: 2174 

DETAILED ACTION 
Specification 

1 . Applicant is reminded of the proper language and format for an abstract of the disclosure. 

The abstract should be in narrative form and generally limited to a single paragraph on a 
separate sheet within the range of 50 to 150 words. It is important that the abstract not exceed 
1 50 words in length since the space provided for the abstract on the computer tape used by the 
printer is limited. The form and legal phraseology often used in patent claims, such as "means" 
and "said," should be avoided. The abstract should describe the disclosure sufficiently to assist 
readers in deciding whether there is a need for consulting the full patent text for details. The 
abstract is less than 50 words. 

Claim Objections 

2. Claims 31-32 are objected to because of the following informalities: in line 10 of page 
25, the word "lest" needs to be changed to — least — . Appropriate correction is required. 

Claim Rejections - 35 USC § 112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

4. Claim 39 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

Claim 39 recites the limitation "said summarization" in line 28 of page 26. There is 
insufficient antecedent basis for this limitation in the claim. 
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Claim Rejections - 35 USC § 102 



5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

6. Claims 1-5, 7-12, 29, 31-33, 42, 43 and 65 are rejected under 35 U.S.C. 102(b) as being 
anticipated by "Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et 
al. ("Kawashima"). 

As per claim 1, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video based upon an event, wherein the 
event is characterized by a start time based upon when the ball is put into play and an end time 
based upon when the ball is considered out of play, where each of the segments includes a 
plurality of frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; e.g. the at bat event 
comprising of a start point in time slightly before the pitching and end point in time slightly after 
the catcher catches the ball if the ball is struck out and after the ball is thrown to a baseman if 
the ball is hit); and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. compressed play). 

As per claim 2, Kawashima teaches a method of processing a video including baseball 
wherein the event is defined by the rules of baseball (pp. 871-873, sections 1.1-2.1.4; events such 
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as scenes in which a batter was struck out or got a hit or a home run is defined by the rules of 
baseball using a spotting technique comprising a search of the minimal warp function by 
comparing input video sequence with pitching/batting model sequences). 

As per claim 3, Kawashima teaches a method of processing a video including baseball 
wherein the start time is temporally proximate a baseball pitch (pg. 872, lines 10-11). 

As per claim 4, Kawashima teaches a method of processing a video including baseball 
wherein the end time is temporally proximate to the batter missing the ball with a bat (pg. 872, 
lines 12-15). 

As per claim 5, Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segment of the video, where each of the segments includes a 
plurality of frames of the video, based upon a series of activities defined by the rules of baseball 
(pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; series of activities such as scenes in which a batter 
was struck out or got a hit or a home run is defined by the rules of baseball using a spotting 
technique comprising a search of the minimal warp function by comparing input video sequence 
with pitching/batting model sequences) that could potentially result in at least one of a score, 
preventing a score, advancing a team toward a score, preventing advancing a team toward a 
score or creating a summarization of the video by including the plurality of segments where the 
summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. compressed play). 

As per claim 7, Kawashima teaches a method of processing a video including baseball 
wherein the activities are determined based upon the color characteristics of the video (pp. 872- 
873, section 2.1.3; activities are spotted by calculating the value from the count of pixels whose 
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intensity change in successive frames are larger than a threshold wherein pixels are 
painted/colored to form an image produced on the screen). 

As per claim 8, Kawashima teaches a method of processing a video including baseball 
wherein the activities are determined based upon scene changes (pp. 872-873; section 1.1-2.1.4; 
wherein an activity such as an at bat activity is a period from a basic scene to the next basic 
scene). 

As per claim 9, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video based upon detecting a play of the 
baseball game, wherein the identifying includes detecting the start of the play and detecting the 
end of the play, where each of the segments includes a plurality of frames of the video (pp. 871- 
873, sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of the play in which a batter was 
struck out or got a hit or a home run is defined by the rules of baseball using a spotting 
technique comprising a search of the minimal warp function by comparing input video sequence 
with pitching/batting model sequences); and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.ka. compressed play). 

As per claim 10, Kawashima teaches a method of processing a video including baseball 
wherein the detecting the end of the play is based upon detecting the start of the play (pp. 872- 
873; section 1.1-2.1.4; wherein a play such as an at bat activity is a period from n end of a basic 
scene to the start of the next basic scene). 
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As per claim 11, Kawashima teaches a method of processing a video including baseball 
wherein the summarization identifies the plurality of segments of the video (pg. 872, section 



As per claim 12, Kawashima teaches a method of processing a video including baseball 
wherein the summarization is a summarized video comprising the plurality of segments 
excluding at least a portion of the video other than the plurality of segments (pg. 872, section 



As per claim 29, Kawashima teaches a method of processing a video including baseball 
comprising: 

identifying a plurality of segments of the baseball video, wherein the identifying for the 
end of at least one of the segments is based upon detecting a scene change, where each of the 
segments includes a plurality of frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2; 
wherein an activity such as an at bat activity is a period from a basic scene to the next basic 
scene); and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than said baseball video (Abstract; pg. 872, section 1.2; 
i.e. the indexed video segments is a digest of the game or summary of the video, a.ka. 
compressed play). 

As per claim 31, Kawashima teaches a method of processing a video including baseball 
wherein the identifying for the start of at least one of the segments is based upon detecting 
regions of color (pp. 872, section 2.1.3; i.e. the identifying for the start of at least one of the 
segments is based upon detecting regions of changing pixels/color). 
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As per claim 32, Kawashima teaches a method of processing a video including baseball 
wherein at least one of the regions is generally brown and at least one of the regions is generally 
green are inherent in a baseball video given that the layout of a baseball field generally consists 
of the colors brown and green (Abstract; pp. 872, section 2.1.3). 

As per claim 33, Kawashima teaches a method of processing a video including baseball 
comprising: 

identifying a plurality of segments of the baseball video, wherein said identifying for the 
end of at least one of said segments is based upon detecting a plurality of scene changes, where 
the first scene change detected after the start of said at least one of said segments is free from 
being selected as said end of said at least one of said segments, where each of the segments 
includes a plurality of frames of the video (pp. 871-873, sections 1.1-2.2; wherein the scene 
change is based upon calculating the value from the count of pixels whose intensity change in 
successive frames are larger than a threshold)', and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than said baseball video (Abstract; pg. 872, section 1.2; 
i.e. the indexed video segments is a digest of the game or summary of the video, a.k.a. 
compressed play). 

As per claim 42, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video, wherein the identifying for at least 
one of the segments includes detecting the start of the segment (pp. 871-873, sections 1.1, 1.2, 
2.1 and 2.2) based, at least in part, upon processing of a frame of the video, where each of the 
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segments includes a plurality of frames of the video wherein the processing characterizes 
whether a batter is sufficiently close to at least one of a catcher and an umpire proximate home 
base (pg. 871, lines 25-27); and 

creating a summarization of the video by including the plurality of segments, where the 
summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. compressed play). 

As per claim 43, Kawashima teaches a method of processing a video including baseball 
wherein at least one of the regions is generally brown and at least one of the regions is generally 
green are inherent in a baseball video given that the layout of a baseball field generally consists 
of the colors brown and green (Abstract; pp. 872, section 2.1.3). 

As per claim 65, Kawashima teaches a method of processing a video comprising 
generating a summary of the video comprising a plurality of segments of the video, where each 
of the segments includes a plurality of frames (Abstract), providing a description of the video, 
associating a portion of the description with a first one of the segments and associating another 
portion of the description with a second one of the segments (pp. 871 and 873, sections 1.1 and 
2.2; provided are indices into scenes). 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21 (2) of such treaty in the English language. 
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8. Claims 47, 48, 54, 59 and 60-64 are rejected under 35 U.S.C. 102(e) as being anticipated 
by "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui"). 

As per claim 47, Rui teaches a method of processing a video including baseball 
comprising identifying a plurality of segment of the video wherein each of the segments includes 
a play of baseball, creating a summarization of the video by including the plurality of segments 
wherein the summarization includes fewer frames than the video (Abstract) and removing at 
least one of the segments from the summary based, at least in part, upon audio information 
related to the at least one of the segments (pg. 105, right column, lines 24-33). 

As per claim 48, Rui teaches a method of processing a video including baseball wherein 
the audio information is obtained exclusively from a temporal analysis (Abstract; pg. 105, right 
column, lines 24-33; 3 rd paragraph, pg. 107 through 3.1.5 Summary, pg. 108; using audio-track 
features, highlights of exciting portions of a baseball video is obtained, so that users can skip the 
boring parts thereby reducing the viewing time). 

As per claim 54, Rui teaches a method of processing a video comprising providing a 
description of the video, identifying a plurality of segments of the video based upon the 
description (pg. 105, "Introduction"; provided are indices/descriptions of the video) and 
generating another video based upon the identified segments, wherein the another video has less 
frames than the video (pg. 105, "Introduction"; a second video comprises of highlights of a first 
video wherein the second video consists of less frames than the first and is based upon the 
exciting portions/identified segments). 

As per claim 59, Rui teaches a method of processing a video comprising a service 
provider identifying: a summary of a video and a description of the summary wherein the 
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summary and the description is provided to a remote user in response to receiving a request from 
the remote user (pg. 105, "Introduction"; a video summary is transmitted in response to user 's 
request via indices/descriptions). 

As per claim 60, Rui teaches a method of processing a video wherein the user request the 
summary with a browser (pg. 105, left column, line 26 through right column, line 2). 

Claims 61 and 63 are individually similar in scope to claim 59 and are therefore rejected 
under similar rationale. 

Claims 62 and 64 are individually similar in scope to claim 60 and are therefore rejected 
under similar rationale. 

Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

10. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Indexing of 
Baseball Telecast for Content-based Video Retrieval", Kawashima et al. ("Kawashima") in view 
of "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui"). 

As per claim 6, although Kawashima teaches a method of processing a video including 
baseball wherein the summarization of the plurality of segments comprises a plurality of 
segments within the video (pg. 872, section 1.2; the indexed video segments of the 
summarization of the plurality of segments is stored as a digest of the game), Kawashima does 
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not explicitly disclose the summarization of the plurality of segments to be in the same temporal 
order as the plurality of segments within the video. Rui teaches a method of processing a video 
including baseball wherein the summarization of the plurality of segments is in the same 
temporal order as the plurality of segments within the video (Abstract; section 5.4; Introduction; - 
a method of allowing users to watch just the highlights of the exciting portions instead of the 
whole game due to time constraints, Le. highlights are extracted automatically so that viewing 
time can be reduced). Therefore, it would have been obvious to an artisan at the time of the 
invention to include Rui's method of processing a video including baseball wherein the 
summarization of the plurality of segments is in the same temporal order as the plurality of 
segments within the video to Kawashima' s method of processing a video including baseball 
wherein the summarization of the plurality of segments comprises a plurality of segments within 
the video so that the time in which sequential plays in a game is being viewed is reduced. 
11. Claims 13-28, 30, 35, 39, 44, 45 and 51-53 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Indexing of Baseball Telecast for Content-based Video Retrieval", 
Kawashima et al. ("Kawashima") in view of "Multimedia Content Analysis", Wang et al. 
("Wang"). 

As per claim 13, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon detecting the start of the play and detecting the end of the play, 
where each of the segments includes a plurality of frames of the video (pp. 871-873, sections 1.1, 
1 .2, 2. 1 and 2.2; e.g. detecting the start of the play in which a batter was struck out or got a hit 
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or a home run is defined by the rules of baseball using a spotting technique comprising a search 
of the minimal warp function by comparing input video sequence with pitching/batting model 
sequences) and wherein the activities are determined based upon the color characteristics of the 
video (pp. 872-873, section 2.1.3; activities are spotted by calculating the value from the count 
of pixels whose intensity change in successive frames are larger than a threshold wherein pixels 
are painted/colored to form an image produced on the screen); and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.ka. compressed play). 

Kawashima does not explicitly disclose one of the region to be of a generally green color and at 
least one region of a generally brown color. Wang teaches a method of processing a video 
comprising identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon detecting at least one region of a color such as green and at 
least one region of another color such as brown (pp. 32-33, "Visual Descriptors"; each segment 
at any level of the hierarchy, a set of audio and visual event-type Ds and annotation DSs, are 
used to characterize the segment or region comprising descriptors such as dominant color, in 
which green and brown is inherently inclusive as dominant colors of baseball, or color 
histogram wherein several types of histograms can be specified including common color 
histogram, which includes the percentage of each quantized color among all pixels in a segment 
or region, or instead of specifying the entire color histogram, one can specify the first few 
coefficients of the Haar transform of the color histogram). Therefore, it would have been 
obvious to an artisan at the time of the invention to include Wang's identifying a plurality of 
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video segments based upon detecting at least one region of a color such as green and at least one 
region of another color such as brown to Kawashima's identifying a plurality of video segments 
based upon color characteristics of the video as an additional visual descriptor in characterizing 
the video segment as part of a goal of video segmentation in automatically grouping shots into 
what are perceived as "scenes". 

As per claims 14-16, the modified Kawashima teaches a method of processing a video 
including baseball comprising detecting at least two regions of the generally green color and/or 
two regions of generally brown color (Kawashima: Abstract; alluding to baseball video 
segmentation Wang: pg. 33, left column, lines 20-26). 

As per claims 17 and 18, the modified Kawashima teaches a method of processing a 
video including baseball comprising color histograms used in identifying a plurality of segments 
of the video wherein the green color and the brown color being generally vertically aligned or 
having sufficient horizontal spatial extent are inherent in a baseball video given that the layout of 
a baseball field comprises of vertical alignment of green and brown colors (Kawashima: 
Abstract; alluding to baseball video segmentation', Wang: pg. 33, lines 6-17; pg. 33, lines 27-46). 

As per claim 19, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon detecting the start of the play and detecting the end of the play, 
where each of the segments includes a plurality of frames of the video (pp. 871-873, sections 1.1, 
1.2,2.1 and 2.2; e.g. detecting the start of the play in which a batter was struck out or got a hit 
or a home run is defined by the rules of baseball using a spotting technique comprising a search 
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of the minimal warp function by comparing input video sequence with pitching/batting model 
sequences) and wherein the activities are determined based upon the color characteristics of the 
video (pp. 872-873, section 2.1.3; activities are spotted by calculating the value from the count 
of pixels whose intensity change in successive frames are larger than a threshold wherein pixels 
are painted/colored to form an image produced on the screen); and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. compressed play). 

Kawashima does not explicitly disclose at least three regions of horizontally oriented regions of 
colors. Wang teaches a method of processing a video comprising identifying a plurality of 
segments of the video, wherein videos are segmented based upon regions of horizontally oriented 
regions of colors (pp. 32-33, "Visual Descriptors"; i.e. color histograms are used in identifying a 
plurality of segments of the video wherein regions of 3 horizontally oriented regions of colors 
are inherent in a baseball video given the layout of a baseball field) Therefore, it would have 
been obvious to an artisan at the time of the invention to include Wang's method of processing a 
video comprising identifying a plurality of segments of the video, wherein videos are segmented 
based upon regions of horizontally oriented regions of colors to Kawashima' s method of 
processing a video comprising identifying a plurality of segments of the video, wherein videos 
are segmented based upon the change in color characteristics of the video as an additional visual 
descriptor in characterizing the video segment as part of a goal of video segmentation in 
automatically grouping shots into what are perceived as "scenes". 
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As per claims 20-22, the modified Kawashima teaches a method of processing a video 
including baseball wherein at least two of the regions have the same dominant color, at least 
three regions have different said colors and at least two regions are located on opposing sides of 
another said detected region of a different color (Kawashima: Abstract; alluding to baseball 
video segmentation; Wang: pg. 33, left column, lines 20-26; described are features that are all 
inherent to the layout of a baseball field). 

As per claim 23, the modified Kawashima teaches a method of processing a video 
including baseball wherein said one of said regions is generally brown and another of the regions 
is generally green are inherent in a baseball video given that the layout of a baseball field 
generally consists of the colors brown and green (Kawashima: Abstract; alluding to baseball 
video segmentation; Wang: pg. 33, left column, lines 20-26). 

As per claim 24, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the end of the 
play, where each of the segments includes a plurality of frames of the video (pp. 871-873, 
sections 1.1, 1.2, 2.1 and 2.2; e.g. detecting the start of the play in which a batter was struck out 
or got a hit or a home run is defined by the rules of baseball using a spotting technique 
comprising a search of the minimal warp function by comparing input video sequence with 
pitching/batting model sequences); and 
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(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. compressed play), 

Kawashima does not explicitly disclose the start of the plurality of segments is identified based 
upon identifying at least three regions of sufficient spatial coherence and sufficient horizontal 
extent. Wang teaches a method of processing a video comprising identifying a plurality of 
segments of the video, wherein videos are segmented based upon shape/spatial geometry such as 
regions of sufficient spatial coherence and horizontal extent (pg. 33, lines 27-46) Therefore, it 
would have been obvious to an artisan at the time of the invention to include Wang's method of 
processing a video comprising identifying a plurality of segments of the video, wherein videos 
are segmented based upon shape/spatial geometry such as regions of sufficient spatial coherence 
and horizontal extent to Kawashima' s method of processing a video including baseball 
comprising identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the end of the 
play in order to provide users with an additional visual descriptor in characterizing the video 
segment as part of a goal of video segmentation in automatically grouping shots into what are 
perceived as "scenes". 

As per claim 25, the modified Kawashima teaches a method of processing a video 
including baseball wherein each said at least three regions is at least one of substantially 
generally green and substantially generally brown are inherent in a baseball video given that the 
layout of a baseball field generally consists of at least three regions of substantially generally 
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green and substantially generally brown (Kawashima: Abstract; alluding to baseball video 
segmentations Wang: pg. 33, left column, lines 20-26). 

As per claim 26, the modified Kawashima teaches a method of processing a video 
including baseball wherein the three regions being generally vertically spaced apart are inherent 
in a baseball video given that the layout of a baseball field comprises of three regions of 
generally vertical spacing (Kawashima: Abstract; alluding to baseball video segmentation', 
Wang: pg. 33, lines 27-46). 

As per claim 27, Kawashima teaches a method of processing a video including baseball 
comprising: 

identifying a plurality of segments of the video based upon an event, wherein the 
identifying for at least one of the segments includes detecting the start of the segment based upon 
processing of a first single frame of the video, where each of the segments includes a plurality of 
frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 2.2); and 

creating a summarization of the video by including the plurality of segments, where the 
summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. compressed play). 

Kawashima does not explicitly disclose verifying that said first single frame is an appropriate 
start of the segment based upon processing of another single frame temporally relevant to the 
first single frame. Wang teaches verifying that said first single frame is an appropriate start of the 
segment based upon processing of another single frame temporally relevant to the first single 
frame (pg. 22, left column, lines 1-8). Therefore, it would have been obvious to an artisan at the 
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time of the invention to include Wang's verifying that said first single frame is an appropriate 
start of the segment based upon processing of another single frame temporally relevant to the 
first single frame to Kawashima's start of the segment based upon processing of another single 
frame temporally relevant to the first single frame in order to reduce errors in segmenting related 
scenes. 

As per claim 28, the modified Kawashima teaches a method of processing a video 
including baseball wherein in said identifying based upon detecting at least one region of 
generally brown and at least one region of generally green are inherent. in a baseball video given 
that the layout of a baseball field generally consists of the colors brown and green (Kawashima: 
Abstract; alluding to baseball video segmentation; Wang: pg. 33, left column, lines 20-26). 

As per claim 30, Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the baseball video, wherein the scene change is 
based upon calculating the value from the count of pixels whose intensity change in successive 
frames are larger than a threshold (pp. 872-873, section 2.1.3). Kawashima does not teach 
identifying the scene change based upon a histogram. Wang teaches a method of processing a 
video wherein identifying the scene change in video segmenting is based upon a histogram 
(Wang: pg. 33, left column, lines 6-17; pg. 33, right column lines 3-12). Therefore, it would have 
been obvious to an artisan at the time of the invention to include Wang's method of processing a 
video wherein scene change in video segmenting is based upon a histogram to Kawashima's 
method of processing a video wherein scene change is based upon calculating the value from the 
count of pixels whose intensity change in successive frames are larger than a threshold in order 
to provide users with an implementation preference. 
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As per claim 35, although Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the baseball video, wherein said 
identifying for the end of at least one of said segments is based upon detecting a plurality of 
scene changes, where each of the segments includes a plurality of frames of the video (pp. 871- 
873, sections 1.1-2.2; wherein the scene change is based upon calculating the value from the 
count of pixels whose intensity change in successive frames are larger than a threshold), 
Kawashima does not explicitly disclose one of the region to be of a generally green color and at 
least one region of a generally brown color. Wang teaches a method of processing a video 
comprising identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon detecting at least one region of a color such as green and at 
least one region of another color such as brown (pp. 32-33, "Visual Descriptors"; each segment 
at any level of the hierarchy, a set of audio and visual event-type Ds and annotation DSs, are 
used to characterize the segment or region comprising descriptors such as dominant color, in 
which green and brown is inherently inclusive as dominant colors of baseball, or color 
histogram wherein several types of histograms can be specified including common color 
histogram, which includes the percentage of each quantized color among all pixels in a segment 
or region, or instead of specifying the entire color histogram, one can specify the first few 
coefficients of the Haar transform of the color histogram). Therefore, it would have been 
obvious to an artisan at the time of the invention to include Wang's identifying a plurality of 
video segments based upon detecting at least one region of a color such as green and at least one 
region of another color such as brown to Kawashima' s identifying a plurality of video segments 
based upon color characteristics of the video as an additional visual descriptor in characterizing 
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the video segment as part of a goal of video segmentation in automatically grouping shots into 
what are perceived as "scenes". 

As per claims 39 and 44, Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the video wherein each of the 
segments includes detecting a play of the baseball game (Abstract) and creating a summarization 
of the video by including the plurality of segments, where the summarization includes fewer 
frames than the video (Abstract; pg. 872, section 1.2; i.e. the indexed video segments is a digest 
of the game or summary of the video, a.ka. compressed play), Kawashima does not explicitly 
disclose detecting a commercial or a segment that has a temporally sufficiently short duration 
within the video and creating a summarization of the video wherein the plurality of segments of 
the video sufficiently temporally close to the commercial are free from being included within the 
summarization, where the segment that has a temporally sufficiently short duration, or 
separating/removing the identified segment from a summarization. Wang teaches a method of 
processing a video comprising identifying a plurality of segments of the video, wherein videos 
are segmented based upon detecting a commercial within the video and creating a summarization 
of the video wherein the plurality of segments of the video sufficiently temporally close to the 
commercial are free from being included within the summarization, where the segment that has a 
temporally sufficiently short duration, or separating/removing the identified segment from a 
summarization (pg. 21, right column; pg. 29, right column; separation of interested video 
portions and commercials). Therefore, it would have been obvious to an artisan at the time of the 
invention to include Wang's method of detecting a commercial within the video and creating a 
summarization of the video wherein the plurality of segments of the video sufficiently 
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temporally close to the commercial are free from being included within the summarization, 
where the segment that has a temporally sufficiently short duration, or separating/removing the 
identified segment from a summarization to Kawashima's method of detecting a play of the 
baseball game in order to provide users with additional criteria in content-based video retrieval. 

As per claim 45, although the modified Kawashima teaches a method of processing a 
video including baseball wherein the commercial is detected based upon a change in color and 
calculation of the most dominant color (Wang: pg. 21, left column, lines 1-1 1; pg. 26, line 33 
through right column, line 10; commercials are detected based upon changes in color, wherein 
color includes black), the modified Kawashima does not explicitly disclose the commercial 
being detected based upon at least one substantially black frame. Official Notice is taken that 
detection of a commercial frame adjacent to a black frame is well known in the art. Therefore, it 
would have been obvious to an artisan at the time of the invention to include detection of a 
commercial frame adjacent to a black frame to the modified Kawashima's method of detection of 
a commercial based upon a change in color and calculation of the most dominant color in order 
to allow for errors in video editing where a black frame is left uncut. 

As per claim 51, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the end of the 
play, where each of the segments includes a plurality of frames of the video (pp. 871-873, 
sections 1 . 1 , 1 .2, 2. 1 and 2.2; e.g. detecting the start of the play in which a batter was struck out 
or got a hit or a home run is defined by the rules of baseball using a spotting technique 
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comprising a search of the minimal warp function by comparing input video sequence with 
pitching/batting model sequences); and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.k.a. compressed play). 

Kawashima does not explicitly disclose the start of the plurality of segments is identified based 
upon detecting different textures. Wang teaches a method of processing a video comprising 
identifying a plurality of segments of the video, wherein videos are segmented based upon 
detecting different textures (pg. 33, "Texture" section) Therefore, it would have been obvious to 
an artisan at the time of the invention to include Wang's method of processing a video 
comprising identifying a plurality of segments of the video, wherein videos are segmented based 
upon detecting different textures to Kawashima' s method of processing a video including 
baseball comprising identifying a plurality of segments of the video, wherein the start of the 
plurality of segments is identified based upon identifying the start of the play and identifying the 
end of the play in order to provide users with an additional visual descriptor in characterizing the 
video segment as part of a goal of video segmentation in automatically grouping shots into what 
are perceived as "scenes". 

As per claim 52, Kawashima teaches a method of processing a video including baseball 
comprising: 

(a) identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the end of the 
play, where each of the segments includes a plurality of frames of the video (pp. 871-873, 
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sections 1.1, 1 .2, 2. 1 and 2.2; e.g. detecting the start of the play in which a batter was struck out 
or got a hit or a home run is defined by the rules of baseball using a spotting technique 
comprising a search of the minimal warp function by comparing input video sequence with 
pitching/batting model sequences)', and 

(b) creating a summarization of the video by including the plurality of segments, where 
the summarization includes fewer frames than the video (Abstract; pg. 872, section 1.2; i.e. the 
indexed video segments is a digest of the game or summary of the video, a.ka. compressed play). 

Kawashima does not explicitly disclose the start of the plurality of segments is identified based 
upon the video being free from substantial translational motion. Wang teaches a method of 
processing a video comprising identifying a plurality of segments of the video, wherein videos 
are segmented based upon the video being free from substantial translational motion (pg. 22, left 
column, 2 nd paragraph). Therefore, it would have been obvious to an artisan at the time of the 
invention to include Wang's method of processing a video comprising identifying a plurality of 
segments of the video, wherein videos are segmented based upon the video being free from 
substantial translational motion to Kawashima' s method of processing a video including baseball 
comprising identifying a plurality of segments of the video, wherein the start of the plurality of 
segments is identified based upon identifying the start of the play and identifying the end of the 
play in order to provide users with an additional visual descriptor in characterizing the video 
segment as part of a goal of video segmentation in automatically grouping shots into what are 
perceived as "scenes". 

As per claim 53, Kawashima teaches a method of processing a video including baseball 
comprising: 
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(a) identifying a plurality of segments of the baseball video, wherein said identifying for 
the end of at least one of said segments is based upon detecting a scene change, where each of 
said segments includes a plurality of frames of the video (pp. 871-873, sections 1.1, 1.2, 2.1 and 
2.2; wherein an activity such as an at bat activity is a period from a basic scene to the next basic 
scene); and 

(b) creating a summarization of the video by including said plurality of segments, where 
the summarization includes fewer frames than said baseball video (Abstract; pg. 872, section 1.2; 
i.e. the indexed video segments is a digest of the game or summary of the video, a.k.a. 
compressed play). 

Kawashima does not explicitly disclose not using the scene change to identify the end of the at 
least on of said segments if the temporally relevant portion of the at least one said segment 
includes substantially horizontal motion. Wang teaches not using the scene change to identify the 
end of the at least on of said segments if the temporally relevant portion of the at least one said 
segment includes substantially horizontal motion (pg. 21, right column, lines 1-2; pg. 22, left 
column, lines 3-35; pg. 23, right column, lines 18-22, lines 34-37 and lines 53-55; pg. 28, right 
column, lines 33-34; pg. 33, lines 13-48). Therefore, it would have been obvious to an artisan at 
the time of the invention to include Wang's method of not using the scene change to identify the 
end of the at least on of said segments if the temporally relevant portion of the at least one said 
segment includes substantially horizontal motion to Kawashima' s method identifying a plurality 
of segments of the baseball video, wherein said identifying for the end of at least one of said 
segments is based upon detecting a scene change so that a desired scene change is not missed. 
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12. Claim 46 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Indexing of 
Baseball Telecast for Content-based Video Retrieval", Kawashima et al. ("Kawashima") in view 
of "Detection of Slow-Motion Replay Segments in Sports Video for Highlights Generation", 
Pan et al. ("Pan"). 

As per claim 46, Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the video wherein each of the segments 
includes a play of baseball wherein the segments include full-speed plays and creating a 
summarization of the video by including the plurality of segments, where the summarization 
includes fewer frames than the video, where a user may select from the summarization including 
only full-speed plays (Abstract; pg. 872, section 1.2; i.e. the indexed video segments is a digest of 
the game or summary of the video, a.ha. compressed play where users may select a full-speed 
play segment among the plurality of segments). Kawashima does not disclose segments that 
include slow motion plays of the full-speed plays and creating a summarization where a user 
may select from the summarization comprising only of slow motions plays. Pan teaches a 
method of processing a video including baseball comprising identifying a plurality of segments 
of the video wherein each of the segments includes a play of baseball ("Introduction", left 
column) wherein the segments include slow motion plays of the full-speed plays ("Introduction", 
right column; in processing the video, slow motion plays of the full-speed plays and full-speed 
plays are identified) and users may select from the summarization comprising only of slow 
motions plays. Therefore, it would have been obvious to an artisan at the time of the invention to 
include Pan's segments that include slow motion plays of the full-speed plays and creating a 
summarization where a user may select from the summarization comprising only of slow 
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motions plays to Kawashima's segments that include full-speed plays and creating a 
summarization where a user may select from the summarization comprising only of full-speed 
plays in order to provide user with the ability to capture inherently important events. 
13. Claim 34 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Indexing of 
Baseball Telecast for Content-based Video Retrieval", Kawashima et al. ("Kawashima") in view 
of "Video summarization using Hidden Markov Model", Huang et al. ("Huang"). 

As per claim 34, although Kawashima teaches a method of processing a video including 
baseball comprising identifying a plurality of segments of the baseball video, wherein said 
identifying for the end of at least one of said segments is based upon detecting a plurality of 
scene changes, where each of the segments includes a plurality of frames of the video (pp. 871- 
873, sections 1.1-2.2; wherein the scene change is based upon calculating the value from the 
count of pixels whose intensity change in successive frames are larger than a threshold), 
Kawashima does not explicitly disclose the frame is identified as primarily the field at the first 
scene change. Huang teaches a method of processing a video including baseball comprising 
identifying a plurality of segments of the baseball video, wherein the frame is identified as 
primarily the field at the first scene change (pg. 473, "Introduction"; e.g. distance view of batter 
shot, defense of the outfield). Therefore, it would have been obvious to an artisan at the time of 
the invention to include Huang's method of processing a video including baseball comprising 
identifying a plurality of segments of the baseball video, wherein the frame is identified as 
primarily the field at the first scene change to Kawashima's method of processing a video 
including baseball comprising identifying a plurality of segments of the baseball video, wherein 
said identifying for the end of at least one of said segments is based upon detecting a plurality of 
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scene changes, where each of the segments includes a plurality of frames of the video in order to 
provide users with additional criteria in content-based video retrieval. 

14. Claims 36-38 and 40-41 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Indexing of Baseball Telecast for Content-based Video Retrieval", Kawashima et al. 
("Kawashima") in view of "Multimedia Content Analysis", Wang et al. ("Wang") and further in 
view of "Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui"). 

As per claim 36, Kawashima teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the video wherein each of the segments 
includes a plurality of frames of the video (Abstract) and creating a summarization of the video 
by including the plurality of segments, where the summarization includes fewer frames than the 
video (Abstract; pg. 872, section 1.2; ue. the indexed video segments is a digest of the game or 
summary of the video, a.k.a. compressed play). Kawashima does not explicitly disclose 
identifying a plurality of segments that are temporally separated by a sufficiently short duration 
and then connecting the identified plurality of segments. Wang teaches a method of processing a 
video comprising identifying a plurality of segments that are temporally separated by a 
sufficiently short duration and then connecting the identified plurality of segments (pg. 21, right 
column; pg. 29, right column; separation of interested video portions and commercials). 
Therefore, it would have been obvious to an artisan at the time of the invention to include 
Wang's method of identifying a plurality of segments that are temporally separated by a 
sufficiently short duration to Kawashima' s method of detecting a play of the baseball game in 
order to provide users with additional criteria in content-based video retrieval. 
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However, the modified Kawashima still does not explicitly disclose connecting the 
identified plurality of segments. Rui teaches a method of processing a video including baseball 
comprising of connecting the identified plurality of segments so that the summary is in the same 
temporal order as the plurality of segments within the video (Abstract; section 5.4; Introduction; 
a method of allowing users to watch just the highlights of the exciting portions instead of the 
whole game due to time constraints, i.e. highlights are extracted automatically so that viewing 
time can be reduced). Therefore, it would have been obvious to an artisan at the time of the 
invention to include Rui's method of processing a video including baseball comprising of 
connecting the identified plurality of segments to the modified Kawashima' s method of 
processing a video including baseball comprising of a plurality of segments within the video so 
that the time in which sequential plays in a game is being viewed is reduced. 

As per claims 37-38 and 40-41, the modified Kawashima teaches a method of processing 
a video including baseball wherein the connecting includes discarding the frames of the video 
between the identified plurality of segments and wherein the connecting results in a single 
segment that includes the identified plurality of segments together with the frames of the video 
between the identified plurality of segments (Wang: pg, 21, right column; pg. 29, right column; 
separation of interested video portions and commercials; Rui: Abstract; section 5.4; 
Introduction; a method of allowing users to watch just the highlights of the exciting portions 
instead of the whole game due to time constraints, i.e. highlights are extracted automatically so 
that viewing time can be reduced). 
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15. Claims 49 and 50 are rejected under 35 U.S.C. 102(e) as being anticipated by 
"Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui") in view of 
"Multimedia Content Analysis", Wang et al. ("Wang"). 

As per claim 49, although Rui teaches a method of processing a video including baseball 
comprising identifying a plurality of segments of the video wherein each of the segments 
includes a play of baseball, creating a summarization of the video by including the plurality of 
segments wherein the summarization includes fewer frames than the video and the duration of at 
least one of the segments from the summary is based, at least in part, upon audio information 
related to the at least on of the segments (Abstract; pg. 105, right column, lines 24-33), Rui does 
not explicitly disclose modifying the duration of at least one of the segments from the summary 
based, at least in part, upon audio information related to the at least on of the segments. Wang 
teaches a method of processing a video comprising identifying a plurality of segments of the 
video, creating a summarization of the video by including the plurality of segments wherein the 
summarization includes fewer frames than the video and modifying the duration of at least one 
of the segments from the summary based, at least in part, upon audio information related to the at 
least on of the segments (pg. 29, left column, lines 49-53; pg. 30, left column, lines 6-22). 
Therefore, it would have been obvious to an artisan at the time of the invention to include 
Wang's modifying the duration of at least one of the segments from the summary based, at least 
in part, upon audio information related to the at least on of the segments to the method of Rui 
wherein the duration of at least one of the segments from the summary based, at least in part, 
upon audio information related to the at least on of the segments in order to provide users with a 
more customized method of processing a video. 
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As per claim 50, the modified Rui teaches a method of processing a video including 
baseball wherein the audio information is obtained exclusively from a temporal analysis (Rui: 
Abstract; pg. 105, right column, lines 24-33; 3 rd paragraph, pg. 107 through 3.1.5 Summary, pg. 
108; using audio-track features, highlights of exciting portions of a baseball video is obtained, so 
that users can skip the boring parts thereby reducing the viewing time), 
16. Claims 55-58 and 66-67 are rejected under 35 U.S.C 103(a) as being unpatentable over 
"Automatically Extracting Highlights for TV Baseball Programs", Rui et al. ("Rui"). 

As per claims 55 and 56, although Rui teaches a method of processing a video wherein 
the video is in accordance with the channel capacity of a transmission medium and of a storage 
device (pg. 105, left column, lines 26-35), Rui does not explicitly disclose the video is 
transcoded in accordance with the channel capacity of a transmission medium and transcoded in 
accordance with the channel capacity of a storage device. Official Notice is taken that 
transcoding video in accordance with the channel capacity of a transmission medium and 
transcoding in accordance with the channel capacity of a storage device is well known in the art. 
Therefore, it would have been obvious to. an artisan at the time of the invention to include a 
method of processing a video wherein the video is transcoded in accordance with the channel 
capacity of a transmission medium and of a storage device to Rui's method of processing a video 
wherein the video is in accordance with the channel capacity of a transmission medium and of a 
storage device so that users may view the segments of video with the associated description in 
various formats. 

As per claims 57 and 58, although Rui teaches a method of processing a video wherein 
descriptions are associated with segments of video (pg. 105, "Introduction"), Rui does not 
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explicitly disclose the description being compliant with MPEG-7 or TV-Anytime standard. 
Official Notice is taken that description of segments of video compliant with MPEG-7 and TV- 
Anytime standards are well known in the art. Therefore, it would have been obvious to an artisan 
at the time of the invention to include description of segments of video compliant with MPEG-7 
and TV-Anytime standards to Rui's associating descriptions with segments of video to provide 
users with additional alternative viewing mediums. 

As per claims 66 and 67, although Rui teaches a method of processing a video wherein 
descriptions are associated with segments of video (pg. 105, "Introduction"), Rui does not 
explicitly disclose the description being compliant with MPEG-7 or TV-Anytime standard. 
Official Notice is taken that description of segments of video compliant with MPEG-7 and TV- 
Anytime standards are well known in the art. Therefore, it would have been obvious to an artisan 
at the time of the invention to include description of segments of video compliant with MPEG-7 
and TV-Anytime standards to Rui's associating descriptions with segments of video to provide 
users with additional alternative viewing mediums. 

Conclusion 

17. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Hoang (US 6,014,183) teaches a method and apparatus for detecting scene changes in a 
digital video stream. 

Syeda-Mahmood (US 6,691,126 Bl) teaches a method and apparatus for locating multi- 
region objects in an image or video database. 
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Bradski et al. (US 6,363,160 Bl) teach an interface using pattern recognition and 
tracking. 

Carlbom et al. (US 6,141,041) teach a method and apparatus for determination and 
visualization of player field coverage in a sporting event. 

Tamir et al. (US 5,923,365) teach sports event video manipulating system for 
highlighting movement. 

Cavallaro et al. (US 6,305,665 Bl) teach a system for determining the end of a path for a 
moving object. 



18. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Examiner Le whose telephone number is (703) 305-7601. The 
examiner can normally be reached on Monday - Friday from 5:30 am to 2:00 pm (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid, can be reached on (703) 308-0640. 

The fax numbers for the organization where this application or proceeding is assigned is 
as follows: 

(703) 872-9306 [Official Communication] 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 
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